Papers by Tae Hwan Jung
Large Product Key Memory for Pretrained Language Models (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing product key memory (PKM) models that increase model capacity with insignificant computational overhead are limited to causal language modeling. |
| Approach: | They propose product key memory (PKM) that enables very efficient and exact nearest neighbor search in a large number of learnable memory slots. |
| Outcome: | The proposed product key memory improves model capacity and performance by replacing a feed-forward network with a model weighted model. |